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Summary 


^  T.-.c  tf.ecry  -1  dynamic  ^rcgrammlng  1b  applied  a  ciaes 
of  problems  Inv  Ivlng  maximization  wver  discrete  sets.  The 
solutKn  is  made  to  depend  on  tne  solution  of  a  class  of 
functional  equations. 
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NOTLS  ON  THE  THEORY  OP  DYNAMIC  PROGRAMMING — IV 
MAXIMIZATION  OVER  DISCRETE  StTS 

Ol  •  Intrv.duL‘11  c  n 

A  pr^bltm  ol  frequent  ;.ccurrence  Is  that  of  determining 
tae  rnuxlmum  of  a  function  P(x  i  .x* ,  .  .  .  ,Xj^)  subject  to  the 
constraints 

(1)  (^)  ^  ^1*  1“1,2,*«»,K 

(b)  e  s^, 

wr.ere  la  a  discrete,  usually  finite,  set.  Tae  Tjost  Important 
case  Is  that  where  each  Is  a  finite  set  of  Integers,  and  an 
Interesting  sub— case  Is  that  where  ■  0  or  1. 

A  particular  class  cf  protlems  cf  tnls  type  concerns 
tf.e  maximization  ol 

N 

(2)  F(x)  =  Z  F.(x.), 

1=1  ^  ^ 

over  the  eet  ol  x^^  c  nctralned  by  tne  relatl-^r;s 

N 

(j)  (s)  ^  G.,(x,)<o’,,  1=1,2, ...,K 

^  w  1  ^  ^  ^ 

(b)  x^  r  Sj  ,  1-1 ,2. ... ,K, 
wlt/i  G.  ,  (x  , )  >  0  f  r  X  C  S  , . 
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iiven  In  the  case  where  the  and  are  linear  functions 
of  the  this  problem  at  the  moment  escapes  any  of  the 
standard  computational  algorithms  of  linear  programming,  such 
as  the  simplex  method  of  0  Dantzlg. 

We  shall  show  that  this  problem  may  be  treated  by  means 
of  thie  functional  equation  technique  of  the  theory  of  dynamic 
programming,  Li]  ,  and  that  this  technique  yields  a  very  simple 
computational  sclurt.cn  whenever  the  number  of  constraints  Is 
small . 

We  shall  also  Indicate  the  application  of  the  method 
to  a  problem  Involving  mutually  exclusive  activities.  Here 
we  have  an  additional  constraint  of  th.e  form 


(m) 


*1*1+1  ■ 


§2 .  Functional  Equation 


Let  us  define  the  sequence  of  functions 


(I) 


fij(ci,c®,.«.,C|*)  ■  Max  Zt  F.(x.), 
^  ^  1-1  ^  ^ 


where  the  Xj^  are  subject  to  the  constraints  of  (1.)).  Then 


(2) 


*  Max  F  1  (x  I ) 

X  j 


wnerc 


(3) 


(a)  G}|(xi)  <  ^1^  (x  I )  ^  c 


K' 


(b)  X,  C  Si. 
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Applyln*:  the  principle  of  optimality,  we  obtain  the  recurrence 
relation 

(^)  fj^(ci,Ca,«..,Cj^)  “Max  t  ^  *”^1N  ^  *  *  *  *  ^K"~^KN  ^  ’ 

*N 

where 

(3)  (^)  ^IN^^N ^  I »  •  •  •  * * 

(b)  e  Sj^. 

§3*  Example 

N 

Conalder  the  problem  of  determining  the  maximum  of  L,j(x)»Z  a.x. 

"  1-1  ^  ^ 

subject  to  the  constraints 

N 

(1)  (a)  I  b-x  <  c, 

1-1  ^  ^ 

(b )  Xj^  -  0  or  1 , 

where  >  0* 

Here 

(2)  f  I  (c )  ■  d I ,  c  >  b  I , 

-  0,  c  <  b  I, 

and 

fj^(c)  -  ^  Max^  ^  ^  ^ 

■  ^N-1^^^ '  c  <  bj^. 


(3) 
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^4 .  PlBcuiBlon 

Tht  functions  msy  now  bs  conputsd  with  ease,  on 

either  a  digital  or  hand  coaputer,  dipending  upon  the  size  of 
the  system,  starting  with  the  known  value  fi(c). 

To  give  an  example,  suppose  that  N  ■  30  and  c  ■  200,  with 

the  a^,  Integers  ranging  between  1  and  10.  The  naive 

^0 

approach  Involves  the  testing  of  2^  sets  of  values,  l.e., 
all  possible  combinations  of  accept  or  reject.  Since  2^^  • 

.  10^*^,  this  Is  a  considerable  task.  Conventional 
linear  progranslng  techniques  fall  because  of  the  restriction 
that  the  be  Integral.  For  the  case  where  N  ■  30,  a  roundoff 
of  the  linear  progranmilng  solution  may  cause  considerable  error. 

Oslng  the  above  method,  we  must  compute  30  functions 
|fj^(c) ,}  each  containing  200  entries,  c  -  1,  2,  3,  If 

the  a^  and  b^  are  Irrational,  we  may  have  to  refine  the  c-grld 
In  order  not  to  Introduce  round-off  errors  of  Importance.  An 
Important  point  to  note  Is  that  doubling  the  size  of  N  will 
double  the  computational  time,  which  Is  to  say  that  the  time 
required  for  computing  the  solution  In  this  fashion  Is  propor¬ 
tional  to  N,  rather  than  dependent  upon  N  In  some  exponential 
fashion,  as  In  ordinary  search  methods. 

In  return  for  tne  labor  expended  In  computing  the  sequence 
|f^(c},j  one  has  all  the  advantages  of  a  "sensitivity  analysis". 
It  Is  easy  to  trace  the  influence  of  c  and  N  upon  the  maximum 
value  and  the  behavior  of  tne  maximizing  x^^  =  x^(c). 
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Let  UB  noM  cUbcubb  In  aore  dttall  the  remark  we  made  In 
the  Introduction  atatlng  that  thia  technique  1b,  at  the  preeent  time, 

restricted  to  problems  Involving  a  saall  number  of  constraints. 

Consider  a  cargo— handling  problem  In  which  we  have  a 
number  of  Items  possessing  values  v^,  weights  w^  and  sizes  s^  . 

We  wish  to  maximize  the  value  of  the  cargo  carried,  subject 
to  weight  restriction  w  and  a  volume  restriction  s. 

The  mathematical  problem  Is  that  of  maximizing 
N 

(1)  L(x)  -  I  *1^-  . 

1-1  ^  ^ 

subject  to  the  restrictions 

N 

(2)  (a)  L  x.w.  <  w, 

1-1  ^  ^  “ 

H 

(b)  Z  x.s.  <  8 

1-1  ^  ^ 

(c)  x^  -  0,  1,  2,  • • • 

■'ef  Inlng 


(3)  L(*)» 

we  readily  obtain 

(^  )  f  i  (w ,  8 )  -  V  ,  lUn  (  L  ~  j  »  sT  ^  ^ ' 

f^(w,  s)  -  Max  ♦  ^k-1^**  “  *k“k* 


s 


9 
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Nhere  R  is  the  set 

(5)  X.  -  0,  1,  2,  •••,  Nln  (  L-i  J,  Li^]  )• 

"k  "k 

Taking  the  paraaeters  e^  and  to  be  integers,  we  will, 
in  general,  be  required  to  N  functions  of  two  variables, 
tabulated  at  the  points  of  A  grid  w  ■  0,  1,  2,  *  *  * ,  V, 

8  ■  0,  1,  2,  •  *  * ,  S.  If  W  and  S  are  of  the  order  of  magnitude 
of  100,  this  requires  10^  values.  This  is  still  within  the 
capability  of  modem  machines. 

It  is  clear,  however,  that  one  additional  constraint  of 
the  same  type  puts  us  in  the  10^  range.  This  exceeds  the 
capability  of  any  present  day  suichine. 

If,  on  the  other  hand,  there  are  a  large  number  of  con¬ 
straints,  each  with  a  small  range,  then  the  method  is  useful. 

§5.  Example — Mutually  Exclusive  Activitiea 

Consider  the  problem  of  the  preceding  section  under  the 
additional  constraint 

(1)  .0,  1  -  1.  2.  •••.  H  -  1 

Define  the  sequence  of  functions 

N 

(2)  b)  -  lUx  Z  a.x, 

"  {xj  1-1  ‘  ^ 

Where  the  x^  are  subject  to  the  constraints 

(3)  (a)  x^.  b  -  0,  b  -  0  or  1 

N 

(b)  Z  b.x.  <  c. 
i-1  ^  ^  ” 


I 
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Th«n  W0  havt  tht  rtcurrtnct  relation 

(^)  f„(c,  ^  C»H»H  *►  ^||_i(®  -  ^k*N' 

I 

To  dtttmlna  tht  solution  wt  aust  compute  the  double  sequence 
[f„{c,  0),  f^{c,  1)}. 
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